Methods, apparatuses, and programs for encoding and decoding picture

ABSTRACT

Compared to conventional intra divided-picture encoding, deterioration in the coding efficiency is suppressed and the encoding computational complexity and the decoding computational complexity are reduced. A divided picture generating unit divides an input encoding target picture into blocks of the same size and generates divided pictures having the same size by collecting pixels having the same relative position within each block. An intra divided-picture encoding processing unit performs intra divided-picture encoding on part of the divided pictures. An inter divided-picture encoding processing unit performs inter divided-picture encoding using another encoded divided picture as a reference picture. When there are a plurality of candidates for the reference picture, a correlation direction calculating unit obtains a combination in which a correlation of a pixel on an original picture is high among combinations of encoded divided pictures and their reference pictures. A reference picture selecting unit selects an encoded divided picture in a direction in which a correlation with an encoding target divided picture is high as the reference picture to be used in the inter divided-picture encoding processing unit.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a 371 U.S. National Stage of InternationalApplication No. PCT/JP2012/082174, filed Dec. 12, 2012. Priority isclaimed on Japanese Patent Application No. 2011-271841, filed Dec. 13,2011; the entire content of both applications is incorporated herein byreference.

TECHNICAL FIELD

The present invention relates to picture encoding and decodingtechnologies, and more particularly, to a picture encoding method, apicture decoding method, a picture encoding apparatus, a picturedecoding apparatus, a picture encoding program, and a picture decodingprogram which realize encoding and decoding capable of reducing thedecoding computational complexity while suppressing degradation in thecoding efficiency as compared to conventional intra-frame predictiveencoding and decoding.

BACKGROUND ART

In H.264, which is an international video coding standard, intra-framepredictive coding has been performed in order to improve a compressionrate in coding using a correlation of pixels between blocks (seeNon-Patent Document 1). This intra-frame prediction is performed inunits of blocks in which some pixels are collected, and three types ofblock sizes of 4×4, 8×8, and 16×16 are available to a luminance signal.In addition, selection from a plurality of prediction modes is possiblefor each block size.

This H.264 uses a method based on extrapolative prediction at the timeof intra-frame prediction, but there is a problem in that the predictionefficiency is low. In order to solve this problem, suppression of blockdistortion using a deblocking filter for an entire frame is performedand thus the computational complexity is increased.

In addition, technology described in Non-Patent Document 2 is known as atechnique of improving the coding efficiency in intra-frame prediction.This technology is a technique in the intra-frame prediction whichsearches encoded areas for a block having a small error and performsencoding using the prediction error therefor with respect to an encodingtarget bock.

FIG. 19 is a flowchart illustrating an example of an intra-framepredictive encoding process in accordance with the conventional art. Inthe intra-frame predictive encoding of Non-Patent Document 2, first, anencoding target picture is divided into N blocks 1 to N having the samesize (step S801). Next, intra-frame predictive encoding is performed onthe first block 1 (step S802). Subsequently, in encoding of blocks 2 andsubsequent blocks, inter-frame predictive encoding is performed using ablock having a small prediction error in encoded areas as a referencepicture and information on a motion vector to the reference picture anda prediction error are encoded (step S803). The process of step S803 isiterated up to the final block N.

The technology proposed in Non-Patent Document 2 is a technique ofimproving the coding efficiency, and a quantization error also tends tobe reduced because it is possible to suppress the occurrence of aprediction error in an area in which the same pattern is iterated. Thus,it is considered possible to reduce the processing complexity of thedeblocking filter.

However, while the method described above may be effective on a picturein which the same pattern is iterated, it is not effective on a picturein which substantially the same pattern does not appear; in this case,neither a prediction error nor a quantization error may be considered tobe significantly reduced. In this case, because it is also impossible toreduce the processing complexity of the deblocking filter, it is notconsidered to be effective in reducing the decoding computationalcomplexity. Furthermore, because it is necessary to transmit offsetvector information representing the relative position of a referenceblock for each block to a decoding end, there is a problem in that acalculation required to decode reference block information also occursin the decoding end and thus the computational complexity is stilllarge.

In order to solve the problem in the technology of Non-Patent Document2, in Non-Patent Document 3, the present inventors et al. have proposedtechnology which reduces the encoding computational complexity and thedecoding computational complexity while suppressing the degradation inthe coding efficiency.

FIG. 20 is a diagram describing the technology proposed in Non-PatentDocument 3. In this proposed technology, in intra-frame encoding of anoriginal picture PIC1, the original picture PIC 1 is separated into fourdivided pictures PIC 10 to PIC 13 having strong correlations betweenpixels at the same position. It is to be noted that squares to whichnumeric values of 0, 1, 2, and 3 are attached in the drawing representpixels. That is, the divided pictures PIC 10 to PIC 13 are set bydividing the original picture PIC 1 of an input encoding target intoblocks each having 2×2 pixels and collecting pixels at the same relativeposition within each block. Intra-frame encoding is performed on onedivided picture PIC 10 thereamong, and predictive encoding is performedby generating a reference picture from an encoded picture in accordancewith a separation method for each of the remaining three dividedpictures PIC11, PIC12, and PIC13.

Although a strong deblocking filter is applied to the first dividedpicture PIC10 on which the intra-frame encoding is performed, thestrength of the deblocking filter is decreased by employing inter-frameencoding having high prediction efficiency for the second to fourthdivided pictures PIC11, PIC12, and PIC13. Thus, it is possible to reducethe computational complexity in a deblocking process as a whole andreduce the decoding computational complexity while maintaining thecoding efficiency.

A processing procedure of the present technique is as follows.

-   (1) The original picture PIC1 is divided into the four divided    pictures PIC 10 to PIC 13 as illustrated in FIG. 20.-   (2) The first divided picture PIC 10 is encoded by intra-frame    encoding.-   (3) A picture is generated by shifting an encoded picture of the    divided picture PIC 10 to the right by a half pixel using a    half-pixel filter.-   (4) Inter-frame encoding is performed on the second divided picture    PIC 11 by determining the picture generated in (3) as a reference    picture and setting a motion vector to 0.-   (5) A picture is generated by shifting the encoded picture of the    divided picture PIC10 down by a half-pixel using the half-pixel    filter.-   (6) Inter-frame encoding is performed on the third divided picture    PIC 12 by determining the picture generated in (5) as a reference    picture and setting a motion vector to 0.-   (7) A picture is generated by shifting an encoded picture of the    divided picture PIC 12 to the right by a half-pixel using the    half-pixel filter.-   (8) Inter-frame encoding is performed on the fourth divided picture    PIC 13 by determining the picture generated in (7) as a reference    picture and setting a motion vector to 0.

This example describes a case in which encoding is performed by dividingthe input original picture PIC 1 of the encoding target into the blockseach having 2×2 pixels, collecting the pixels at the same relativeposition within each block, and setting the four divided pictures PIC 10to PIC13. However, to further generalize, the divided pictures may beset by dividing the original picture into blocks each having n×m pixelsand rearranging a plurality of pixel groups (here referred to assub-blocks) at the same relative position in each block. A sub-block hasn₁×m₁ pixels (where 1≦n₁<n and 1≦m₁<m).

In the conventional technique, encoding is performed as follows. First,one or more pixels (sub-blocks) are extracted from an input picture atequal intervals, a plurality of divided pictures are generated bycollecting these sub-blocks, and intra divided-picture encoding in whichat least one divided picture is encoded using only the divided pictureis performed. In encoding of the other divided pictures, interdivided-picture predictive encoding is performed using an encodeddivided picture. That is, in accordance with the relative positionalrelationship between a pixel included in an encoding target dividedpicture and a pixel included in an encoded divided picture based on areference picture using the encoded divided picture as the referencepicture, a predicted picture is generated by, for example, applying, tothe reference picture, a filter as used when an interpolation picture ofdecimal pixel accuracy is generated and an error signal between thepredicted picture and the encoding target divided picture is encoded.

FIG. 21 is a flowchart of a process of a conventional technique.

First, in step S900, divided pictures P0 to PN are generated by dividinga picture into blocks of the same size. Next, in step S901, intradivided-picture encoding is performed on some divided pictures P0 to PM(where 0≦M<N) among the generated divided pictures P0 to PN.Subsequently, in step S902, inter divided-picture encoding is performedon divided pictures P(M+1) to PN using an encoded block as a referencepicture.

PRIOR ART DOCUMENTS Non-Patent Documents

Non-Patent Document 1: ITU-T Rec. H.264, “Advanced video coding forgeneric audiovisual services,” March 2005.

Non-Patent Document 2: J. Yang, B. Yin, Y. Sun, and N. Zhang, “Ablock-matching based intra frame prediction for H.264/AVC,” inProceedings of IEEE International Conference on Multimedia and Expo(ICME '06), pp. 705-708, Toronto, Canada, July 2006.

Non-Patent Document 3: Mayuko Watanabe, Masaki Kitahara, AtsushiShimizu, Hirohisa Jozawa: “A Study on low complexity decoding of intracoding,” Proceedings of the 2011 Institute of Electronics, Information,and Communication Engineers (IEICE) General Conference, D-11-39, March2011.

SUMMARY OF INVENTION Problems to be Solved by the Invention

In a decoding process in video coding such as H.264, which is aninternational video coding standard, a deblocking filter which is usedto reduce block distortion occupies a large proportion of the processingcomplexity. If a method for performing predictive encoding betweendivided pictures obtained by extracting and rearranging pixels or pixelgroups in accordance with a given rule proposed in Non-Patent Document 3is used to address this problem, it is possible to suppress thegeneration of block distortion. This method makes it possible to reducethe computational complexity of the deblocking filter because the numberof positions to which the deblocking filter is applied is reduced.

However, in this method, there is room for improvement with respect tothe following points.

-   (1) First, when two or more reference pictures are used in inter    divided-picture encoding, a reference picture index representing a    reference picture used in predictive encoding should be encoded in    order to identify a reference picture to be used during decoding,    and thus a bit amount is increased. Thus, the coding efficiency    deteriorates. In contrast, if the direction of the reference picture    is fixed, the prediction accuracy is decreased and the coding    efficiency deteriorates.-   (2) In addition, there is a problem in that the bit amount is    further increased because a reference picture index should be sent    to the decoding end for every block when the reference picture is    switched for every block within the divided pictures, and the bit    amount is increased due to encoding of reference picture indices.

An object of the present invention is to improve the coding efficiencyby improving the above-described inter divided-picture encoding andmaking encoding of a reference picture index unnecessary in the interdivided-picture encoding.

Means for Solving the Problems

The most significant feature of the present invention is that, in apicture encoding scheme of performing a process of intra-framepredictive encoding (the same is also applied to decoding) by intradivided-picture encoding and inter divided-picture encoding usingdivided pictures obtained by rearranging pixels (or pixel groups), whena reference picture to be referred to in the inter divided-pictureencoding is selected, an encoded divided picture to which a pixel in adirection having a high correlation with a pixel on an original pictureof an encoding target divided picture belongs is obtained and determinedas the reference picture. In addition, it is unnecessary to encode areference picture index representing the encoded divided pictureselected as the reference picture using a common selection logic betweenan encoder (picture encoding apparatus) and a decoder (picture decodingapparatus) in selection of the reference picture.

The present invention performs the following process in compressionencoding of an input picture.

-   (1) Divided pictures of the same size are generated by dividing the    input picture into blocks each having n×m pixels, dividing each    divided block into sub-blocks each having n₁×m₁ pixels (where 1≦n₁<n    and 1≦m₁<m), and collecting sub-blocks at the same relative position    in the blocks.-   (2) Intra divided-picture encoding is performed on at least one of    the divided pictures. Here, the intra divided-picture encoding is    encoding by intra-frame prediction using a divided picture as a    frame unit.-   (3) In order to encode a divided picture other than the divided    picture subjected to the intra divided-picture encoding, an encoded    divided picture having a short distance on the original picture with    respect to pixels at the same position in the encoding target    divided picture and encoded divided pictures is selected as a    reference picture to be used in inter divided-picture predictive    encoding of an encoding target divided picture. When there are a    plurality of candidates for the reference picture, an encoded    divided picture to which a pixel in a direction having a high    correlation with a pixel of the encoding target divided picture    belongs is obtained and selected as the reference picture. That is,    the encoded divided picture having the high correlation is used as    the reference picture. Whether the correlation is high is determined    from, for example, prediction errors of the encoded divided picture.-   (4) A predicted picture for the encoding target divided picture is    generated using the selected reference picture, and inter    divided-picture predictive encoding is performed. In this inter    divided-picture predictive encoding, for example, inter    divided-picture predictive encoding is performed using, as the    predicted picture, a picture obtained by applying a predetermined    filter determined in accordance with the relative position between    corresponding pixels of the encoding target divided picture and the    reference picture on the original picture to the reference picture.    Here, the inter divided-picture predictive encoding is encoding by    inter-frame prediction which is performed using each divided picture    as a frame unit.-   (5) An encoded bitstream is output by performing information source    encoding on encoding results by the above intra divided-picture    encoding and inter divided-picture predictive encoding.

The selection of the reference picture in the above-described process(3) can be performed for each of areas (corresponding to macroblocks orthe like of H.264) each having n₂×m₂ pixels obtained by dividing theencoding target divided picture. Here, this area is referred to as adivided picture block. That is, by selecting a reference picture havinga high correlation for every divided picture block of an encodingtarget, inter divided-picture predictive encoding in which the referencepicture is switched to an optimum one in one encoding target dividedpicture can be performed.

The following are two methods as a process of obtaining an encodeddivided picture to which a pixel in a direction having a highcorrelation belongs in the above-described process (3).

In a first method, a sum of prediction errors between a decoded pictureof an encoded divided picture serving as a candidate for a referencepicture and a predicted picture created from a reference picture of theencoded divided picture is calculated for each candidate for thereference picture. Then, a combination of the encoded divided pictureand the reference picture in which the sum of the prediction errors issmall is obtained and a direction connecting pixels on an originalpicture of corresponding pixels within these pictures is determined asthe direction in which the correlation is high. An encoded referencepicture to which a pixel in the direction having the high correlationbelongs for the encoding target divided picture is determined as areference picture to be used in encoding of an encoding target dividedpicture.

In a second method, instead of generating the predicted picture for theencoded divided picture and calculating the errors as in the firstmethod, a sum of prediction errors already present as encoded data bythe inter divided-picture encoding is calculated, a correlation isdetermined from the sum of the prediction errors, and a referencepicture is selected, thereby suppressing an increase in the decodingcomputational complexity.

In the above-described first method, in order to calculate the directionin which the correlation is high, the differences (errors) between thedecoded picture of the picture subjected to the inter divided-pictureencoding and the predicted picture created from its reference pictureare calculated. However, when the predicted picture is regenerated forevery divided picture or every divided picture block and a differencefrom the decoded picture of the picture subjected to the divided-pictureencoding is calculated, the computational complexity is significantlyincreased.

In contrast, in the second method, it is possible to significantlyreduce an increase in the encoding/decoding computational complexitywhile maintaining the effect substantially equal to that of the firstmethod by calculating the sum of the prediction errors using theprediction errors generated as encoded data in advance and determiningthe correlation.

In addition, in the present invention, the following process is carriedout in performing decoding on encoded data of a picture encoded by theabove-described method.

-   (1) Encoded data obtained by dividing an input picture into blocks    each having n×m pixels, dividing each divided block into sub-blocks    each having n₁×m₁ pixels (where 1≦n₁<n and 1≦m₁<m), collecting    sub-blocks at the same relative position within the blocks to    generate divided pictures of the same size, and performing encoding    in a picture encoding apparatus is input and information source    decoding is performed thereon.-   (2) Intra divided-picture decoding is performed on at least one of    the divided pictures from decoded data.-   (3) When a divided picture other than the divided picture subjected    to the intra divided-picture decoding is decoded, a decoded divided    picture having a short distance on the original picture with respect    to pixels at the same position in the decoding target divided    picture and decoded divided pictures is selected as a reference    picture to be used in inter divided-picture predictive decoding for    a decoding target divided picture. When there are a plurality of    candidates for the reference picture, a decoded divided picture to    which a pixel in a direction having a high correlation with a pixel    of the decoding target divided picture belongs is obtained and    selected as the reference picture. That is, the decoded divided    picture having the high correlation is used as the reference picture    for the decoding target divided picture. Whether the correlation is    high is determined from, for example, prediction errors of the    decoded divided picture.-   (4) A predicted picture for the decoding target divided picture is    generated using the selected reference picture and inter    divided-picture predictive decoding is performed.-   (5) A decoded picture is configured by arranging each pixel in each    divided picture at an original position in the original picture from    the divided pictures decoded by the intra divided-picture decoding    and the inter divided-picture predictive decoding.

The selection of the reference picture in the above-described process(3) can be performed for each of divided picture blocks each havingn₂×m₂ pixels obtained by dividing the decoding target divided picture.

In addition, a method similar to the first or second method at the timeof encoding described above is used in a process of obtaining a decodeddivided picture to which a pixel in a direction having a highcorrelation belongs in the above-described process (3).

The operation of the present invention is as follows. In theconventional inter divided-picture encoding method, it is necessary tocreate and encode a reference picture index representing an encodeddivided picture used as a reference picture. In the present invention,overhead of the reference picture index is eliminated by employing adivided picture having a high correlation as a reference picture insteadof creating and encoding the reference picture index. Thereby, thecoding efficiency is improved.

Details thereof are as follows. In the conventional art, selection froma plurality of reference pictures is possible and it is necessary to,for example, measure a square error between a predicted picture capableof being created from each reference picture and an encoded dividedpicture, select a reference picture having a small square error, andencode its reference picture index, in order to select a referencepicture having high coding efficiency from among the reference pictures.In particular, when the reference picture is switched for every dividedpicture block, a bit amount of the reference picture index occurs forevery divided picture block and the bit amount is increased.

In the present technique, using the fact that there is a strong spatialcorrelation between divided pictures which are originally one originalpicture, a correlation between a picture which has been subjected tointer divided-picture encoding and is to be used as a reference pictureand a reference picture of the encoded picture is estimated using errorswith a predicted picture. A direction in which the errors are small isestimated as a direction in which the correlation is highest, and thedirection is used as a reference direction in an encoding target dividedpicture. Because the strength of the correlation depending on thedirection is considered to be constant between adjacent pixels in theoriginal picture, even when information specifying a reference picturesuch as a reference picture index is not sent to a decoder end, it ispossible to suppress a deterioration in improvement in the codingefficiency by encoding prediction errors using a picture in thereference direction viewed from the encoding target divided picture as areference picture.

In particular, in the present technique, it is not necessary to encodeinformation specifying the reference picture such as the referencepicture index because both an encoder and a decoder can select the samereference picture in accordance with the same process using onlyinformation of encoded/decoded pictures and it is possible to reduce abit amount therefor.

Advantageous Effects of the Invention

In accordance with the present invention, in intra-frame predictiveencoding using a method for generating divided pictures divided byextracting pixels or a pixel group from an encoding target picture andperforming intra divided-picture encoding and inter divided-pictureencoding on the divided pictures, it is possible to select anappropriate reference picture to be used in the inter divided-pictureencoding and it is unnecessary to encode a reference picture indexrepresenting a reference picture. Thus, it is possible to improve thecoding efficiency and reduce the computational complexity involved in adeblocking filter process.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an example of a configuration of apicture encoding apparatus.

FIG. 2A is a diagram illustrating an example of generating dividedpictures by a divided picture generating unit.

FIG. 2B is a diagram illustrating an example of generating the dividedpictures by the divided picture generating unit.

FIG. 2C is a diagram illustrating an example of generating the dividedpictures by the divided picture generating unit.

FIG. 2D is a diagram illustrating an example of generating the dividedpictures by the divided picture generating unit.

FIG. 3 is a diagram illustrating an example of generating a predictedpicture by an inter divided-picture encoding processing unit.

FIG. 4 is a flowchart of a picture encoding process.

FIG. 5 is a detailed flowchart of an inter divided-picture encodingprocess (example 1).

FIG. 6 is a diagram illustrating an example (example 1) of a detailedconfiguration of the picture encoding apparatus.

FIG. 7 is a detailed flowchart of an inter-divided picture encodingprocess (example 2).

FIG. 8 is a diagram illustrating an example (example 2) of a detailedconfiguration of the picture encoding apparatus.

FIG. 9 is a diagram illustrating an example of division of an encodingtarget picture.

FIG. 10A is a diagram illustrating an example of a reference pictureselecting method.

FIG. 10B is a diagram illustrating an example of the reference pictureselecting method.

FIG. 10C is a diagram illustrating an example of the reference pictureselecting method.

FIG. 10D is a diagram illustrating an example of the reference pictureselecting method.

FIG. 11 is a diagram illustrating an example of a configuration of apicture decoding apparatus.

FIG. 12 is a flowchart of a picture decoding process.

FIG. 13 is a diagram illustrating an example (example 1) of a detailedconfiguration of the picture decoding apparatus.

FIG. 14 is a diagram illustrating an example (example 2) of the detailedconfiguration of the picture decoding apparatus.

FIG. 15 is a diagram illustrating an example of a moving-pictureencoding apparatus to which the present invention is applicable.

FIG. 16 is a diagram illustrating an example of a moving-picturedecoding apparatus to which the present invention is applicable.

FIG. 17 is a diagram illustrating an example of a configuration ofhardware when the picture encoding apparatus is realized using asoftware program.

FIG. 18 is a diagram illustrating an example of a configuration ofhardware when the picture decoding apparatus is realized using asoftware program.

FIG. 19 is a flowchart illustrating an example of an intra-framepredictive encoding process in accordance with the conventional art.

FIG. 20 is a diagram illustrating an example of a conventionalintra-picture predictive encoding process using divided picturesobtained by extracting pixels at fixed intervals.

FIG. 21 is a flowchart of the conventional intra-picture predictiveencoding process using the divided pictures obtained by extracting thepixels at the fixed intervals.

MODES FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described withreference to the drawings.

[Picture Encoding Apparatus]

FIG. 1 is a diagram illustrating an example of a configuration of thepicture encoding apparatus. The picture encoding apparatus 10 includes adivided picture generating unit 11, an intra divided-picture encodingprocessing unit 12, an inter divided-picture encoding processing unit13, a correlation direction calculating unit 15, a reference pictureselecting unit 16, and an information source encoding unit 14.

The divided picture generating unit 11 divides an input picture intoblocks each having n×m pixels, divides each divided block intosub-blocks each having n₁×m₁ pixels (where 1≦n₁<n and 1≦m₁<m), andcollects sub-blocks at the same relative position within the blocks togenerate divided pictures of the same size.

FIGS. 2A to 2D are diagrams each illustrating an example of generatingdivided pictures by the divided picture generating unit 11. For example,the divided picture generating unit 11 uses an original pictureillustrated in FIG. 2A as the input picture and divides the originalpicture into blocks Mj (j=0, 1, . . . , J) each having n×m pixels asillustrated in FIG. 2B. Next, the divided picture generating unit 11divides each block Mj into sub-blocks Bjk (k=0, 1, . . . , K) eachhaving n₁×m₁ pixels (where 1≦n₁<n and 1≦m₁<m) as illustrated in FIG. 2C.

Next, as illustrated in FIG. 2D, the divided picture generating unit 11generates divided pictures Pk (k=0, 1, . . . , K) of the same size bycollecting sub-blocks Bjk at the same relative position within theblocks from each block Mj. A divided picture P0 is a collection ofsub-blocks B00, B10, . . . , and BJ0, a divided picture P1 is acollection of sub-blocks B01, B11, . . . , and BJ1, . . . , and adivided picture PK is a collection of sub-blocks B0K, B1K, . . . , andBJK.

The intra divided-picture encoding processing unit 12 performs intradivided-picture encoding on some divided pictures (which may be only afirst divided picture) including the first divided picture generated bythe divided picture generating unit 11. Here, any encoding method ofperforming encoding using only pixel information of a divided pictureserving as a current encoding target without referring to other dividedpictures may be used as the intra divided-picture encoding. For example,it is possible to use a method such as intra-predictive encoding in anH.264 coding scheme.

The inter divided-picture encoding processing unit 13 performs interdivided-picture encoding on a divided picture that is not yet encodedamong the divided pictures generated by the divided picture generatingunit 11. In this inter divided-picture encoding, an encoded dividedpicture is used as a reference picture, and a predicted picture isgenerated by applying, to the reference picture, a predetermined filterdetermined by the relative position between corresponding pixels on theoriginal picture of a divided picture serving as a current encodingtarget and the reference picture. Errors between the predicted pictureand the encoding target divided picture are encoded and its encodedinformation is sent to the information source encoding unit 14.

The information source encoding unit 14 performs entropy encoding on theencoded information which is outputs of the intra divided-pictureencoding processing unit 12 and the inter divided-picture encodingprocessing unit 13, and outputs encoded data.

The present embodiment is particularly different from interdivided-picture predictive encoding of the conventional art as shown inNon-Patent Document 3 in that the correlation direction calculating unit15 which inspects a correlation on the original picture between a pixelof an encoding target picture and a pixel of an encoded picture whenthere are a plurality of candidates for the reference picture and thereference picture selecting unit 16 which selects an encoded dividedpicture to which a pixel in a direction having a high correlationbelongs as the reference picture are provided.

The correlation direction calculating unit 15 obtains a divided picturehaving the smallest sum of absolute values or the smallest sum ofsquares of prediction errors among encoded divided pictures serving ascandidates for the reference picture, determines, from its result, adirection of corresponding pixels of the encoded divided picture and itsreference picture on the original picture as a direction in which thecorrelation is high, and notifies the reference picture selecting unit16 of the correlation direction.

The reference picture selecting unit 16 selects an encoded dividedpicture in the correlation direction calculated by the correlationdirection calculating unit 15 as the reference picture for the encodingtarget divided picture, and notifies the inter divided-picture encodingprocessing unit 13 of the reference picture.

FIG. 3 is a diagram illustrating an example of generating a predictedpicture in the inter divided-picture encoding processing unit 13.Hereinafter, an example of generating a predicted picture in which adivided picture Pi is an encoded divided picture serving as thereference picture and a divided picture Pk is an encoding target dividedpicture on which inter divided-picture predictive encoding is performedwill be described. A sub-block belonging to the divided picture Pi isrepresented as Bi and a sub-block belonging to the divided picture Pk isrepresented as Bk.

Assuming a positional relationship in the original picture between thesub-block Bi of the divided picture Pi and the sub-block Bk of thedivided picture Pk as illustrated in FIG. 3(A), sub-blocks Bi positionedin the vicinity of the sub-block Bk are extracted as illustrated in FIG.3(B). In this example, two sub-blocks Bi are extracted for one sub-blockBk, but the number of sub-blocks to be extracted is not limited to 2.Next, as illustrated in FIG. 3(C), a pixel value of a sub-block Bk′ iscalculated by applying an interpolation filter to pixel values of thetwo extracted sub-blocks Bi. Filter coefficients predetermined by therelative position between the sub-block Bi and the sub-block Bk on theoriginal picture is used as filter coefficients of the interpolationfilter. It is to be noted that various conventional methods are known asan interpolation method by an interpolation filter, and a predictedpicture may be generated using any interpolation method.

A collection of sub-blocks Bk′ generated by the interpolation asdescribed above is determined as a predicted picture to be used in interdivided-picture predictive encoding of the divided picture Pk.

[Flow of Picture Encoding Process]

FIG. 4 is a flowchart of the picture encoding process. The flow of thepicture encoding process will be described in accordance with FIG. 4.

First, the divided picture generating unit 11 generates divided picturesP0 to PN by dividing an input picture into blocks of the same size andcollecting sub-blocks (pixels or pixel groups) at the same relativeposition in each block (step S101).

Next, the intra divided-picture encoding processing unit 12 performsintra divided-picture encoding on some divided pictures P0 to PM (whereM<N) among the divided pictures P0 to PN (step S102). Here, it isdesirable that positions of the sub-blocks constituting a dividedpicture serving as a target of the intra divided-picture encoding on theoriginal picture be separated at predetermined intervals. Only the firstdivided picture P0 may be the target of the intra divided-pictureencoding.

Next, the inter divided-picture encoding processing unit 13 performsinter divided-picture encoding on the divided pictures P(M+1) to PL(where M<L<N) using an encoded divided picture as a reference picture(step S103). Each of the divided pictures P(M+1) to PL is, for example,a divided picture when the number of candidates for the referencepicture is one, such as a divided picture that is not yet encoded and isadjacent on the right-hand side of a divided picture subjected to theintra divided-picture encoding or a divided picture that is not yetencoded and is directly below the divided picture subjected to the intradivided-picture encoding. Here, the number of the divided picture onwhich the inter divided-picture encoding is performed can be determinedin advance.

Subsequently, the inter divided-picture encoding processing unit 13performs inter divided-picture encoding on the remaining dividedpictures P(L+1) to PN using an encoded divided picture to which a pixelin a direction having a high correlation with respect to an encodingtarget divided picture belongs selected by the correlation directioncalculating unit 15 and the reference picture selecting unit 16 as thereference picture (step S104).

[Detailed Flow (Example 1) of Inter Divided-Picture Encoding Process]

FIG. 5 illustrates the first example of the detailed processingflowchart of step S104 illustrated in FIG. 4.

In step S201, a process from step S202 to step S208 is iterated on eachdivided picture Pi (i is (L+1) to N).

In step S202, each divided picture Pi is divided into divided pictureblocks of encoding units and a process from step S203 to step S207 isiterated for every divided picture block. Although this divided pictureblock corresponds to a macroblock or the like in H.264 coding, such as16×16 pixels, the size of the divided picture block may be arbitrarilyset.

In step S203, for a divided picture block of an encoding target, apredicted picture B is created from a decoded picture A of a dividedpicture subjected to inter divided-picture encoding and its referencepicture, prediction errors which are the difference between the decodedpicture A and the predicted picture B are calculated, and a sum ofabsolute values or a sum of squares (hereinafter, the sum of absolutevalues or the sum of squares is simply referred to as a “sum”) of theprediction errors is obtained.

In step S204, a combination of the encoded divided picture (referred toas PA) and its reference picture (referred to as PB) in which the sum ofthe prediction errors between the decoded picture A and the predictedpicture B obtained in step S203 is small is obtained.

In step S205, a direction connecting two points of point A (X(PA),Y(PA)) and point B (X(PB), Y(PB)) on the original picture is determinedas a direction in which the correlation is high for the combination ofthe encoded divided picture PA and the reference picture PB obtained instep S204.

In step S206, for the divided picture block of the encoding target, anencoded divided picture to which a pixel in the direction having thehigh correlation belongs is selected as a reference picture, a predictedpicture is generated from the reference picture, and interdivided-picture encoding is performed.

In step S207, a determination as to whether processing of all dividedpicture blocks within the divided picture Pi has ended is made; if thereis a divided picture block which has not yet been processed, a processfrom step S202 is iterated for the divided picture block.

In step S208, a determination as to whether all divided pictures P(L+1)to PN have been encoded is made, and the process from step S201 isiterated until all of the divided pictures P(L+1) to PN are encoded.

[Example (Example 1) of Detailed Configuration of Picture EncodingApparatus]

FIG. 6 illustrates the example of the detailed configuration of thepicture encoding apparatus 10 illustrated in FIG. 1. The pictureencoding apparatus 10 illustrated in FIG. 6 is an example of aconfiguration of an apparatus which executes a process of the firstexample described with reference to FIG. 5. Because the divided picturegenerating unit 11, the information source encoding unit 14, and thereference picture selecting unit 16 in FIG. 6 correspond to those of thesame reference numerals illustrated in FIG. 1, a description thereof isomitted.

An intra divided-picture encoding unit 101 performs intradivided-picture encoding on divided pictures P0 to PM. An intradivided-picture decoding unit 102 decodes a divided picture encoded bythe intra divided-picture encoding unit 101 and stores the decodedpicture in a decoded picture memory 103. In the decoded picture memory103, a decoded picture of a divided picture subjected to interdivided-picture encoding is also stored later.

In order to perform inter divided-picture predictive encoding on adivided picture that is not yet encoded among divided pictures generatedby the divided picture generating unit 11, a predicted picturegenerating unit 104 generates a predicted picture using an encodeddivided picture within the decoded picture memory 103 as a referencepicture for every divided picture block (hereinafter, the dividedpicture block may be simply referred to as a divided picture) of thedivided picture. In generation of the predicted picture, a predictedpicture is generated by applying, to the reference picture, apredetermined filter determined by the relative position betweencorresponding pixels of the divided picture serving as a currentencoding target and the reference picture on the original picture.

A difference calculating unit 105 calculates prediction errors bysubtracting each pixel value of the predicted picture generated by thepredicted picture generating unit 104 from each pixel value of thedivided picture block serving as the current encoding target. Aprediction error encoding unit 106 performs an orthogonal transform anda quantization process on the calculated prediction errors to encode theprediction errors.

The information source encoding unit 14 performs entropy encoding onencoded information of the intra divided-picture encoding unit 101 andencoded information of the prediction error encoding unit 106 encoded byinter divided-picture predictive encoding and outputs encoded data.

In the first example, a prediction error decoding unit 107 decodes theprediction errors encoded by the prediction error encoding unit 106. Ina picture decoding unit 108, an adder 109 adds the prediction errorsdecoded by the prediction error decoding unit 107 to the predictedpicture generated by the predicted picture generating unit 104 to decodean inter divided-picture encoded picture. It is to be noted that, in thepicture decoding unit 108, a post-processing filter such as a deblockingfilter may be applied after the predicted picture is added to theprediction errors. The decoded picture of the divided picture, which hasbeen decoded, is stored in the decoded picture memory 103.

A subtractor 110 calculates a difference between the decoded picture ofthe encoded divided picture and the predicted picture, and a predictionerror calculating unit 111 calculates a sum of prediction errors forevery encoded divided picture serving as a candidate for the referencepicture. A prediction error comparing unit 112 obtains a divided picturehaving the smallest sum of the prediction errors calculated by theprediction error calculating unit 111, determines, from the result, adirection of corresponding pixels of the encoded divided picture havingthe smallest sum of the prediction errors and its reference picture onthe original picture as a direction in which the correlation is high,and notifies the reference picture selecting unit 16 of the correlationdirection.

The reference picture selecting unit 16 selects an encoded dividedpicture in the correlation direction calculated by the prediction errorcomparing unit 112 from the decoded picture memory 103 as the referencepicture for the divided picture of the encoding target, and notifies thepredicted picture generating unit 104 of the reference picture.

[Detailed Flow (Example 2) of Inter Divided-Picture Encoding Process]

FIG. 7 illustrates the second example of the detailed processingflowchart of step S104 illustrated in FIG. 4.

In the second example, the process of step S303 is different from thatof the first example, and the other steps S301, S302, and S304 to S308are the same as steps 201, S202, and S204 to S208 of the first exampledescribed with reference to FIG. 5.

In step S303, a sum of prediction errors between the decoded picture Aof the divided picture subjected to inter divided-picture encoding andits predicted picture B is obtained for a divided picture block of anencoding target. That is, in step S303, instead of regenerating thepredicted picture for every divided picture block, generating thedecoded picture A, and obtaining the sum of the prediction errors, thesum of prediction errors is calculated by directly using the predictionerrors already generated as encoded data and the correlation isdetermined. Thereby, an increase in the computational complexity of adecoding calculation is suppressed.

[Example (Example 2) of Detailed Configuration of Picture EncodingApparatus]

FIG. 8 illustrates the second example of the detailed configuration ofthe picture encoding apparatus 10 illustrated in FIG. 1. The pictureencoding apparatus 10 illustrated in FIG. 8 is an example of aconfiguration of an apparatus which executes the process of the secondexample described with reference to FIG. 7. Because the components ofFIG. 8 having the same reference numerals as those of the pictureencoding apparatus 10 of the first example illustrated in FIG. 6described above have the same functions as those illustrated in FIG. 6,a detailed description thereof is omitted.

In the case of the second example, a prediction error decoding unit 120decodes prediction errors by applying an inverse quantization processand an inverse orthogonal transform on the prediction errors encoded bythe prediction error encoding unit 106. A prediction error calculatingunit 121 calculates a sum of the prediction errors decoded by theprediction error decoding unit 120 for every divided picture. Aprediction error comparing unit 122 obtains a divided picture having thesmallest sum of the prediction errors calculated by the prediction errorcalculating unit 121, determines, from the result, a direction ofcorresponding pixels of the encoded divided picture having the smallestsum of the prediction errors and its reference picture on the originalpicture as a direction in which the correlation is high, and notifiesthe reference picture selecting unit 16 of the correlation direction.

The reference picture selecting unit 16 notifies a picture decoding unit123 of the fact that an encoded divided picture in the notifiedcorrelation direction is determined as the reference picture for theencoding target divided picture from the result of the prediction errorcomparing unit 122. The picture decoding unit 123 generates a decodedpicture of the notified encoded divided picture from its predictedpicture and the prediction errors, and stores it in the decoded picturememory 103. The predicted picture generating unit 104 generates apredicted picture used to encode the encoding target divided pictureblock using a decoded picture stored in the decoded picture memory 103as a reference picture.

[Specific Example of Encoding]

FIG. 9 illustrates an example of division of an encoding target picture.In the example described below, it is assumed that the divided picturegenerating unit 11 divides one frame of an input picture which is anencoding target into blocks M0, M1, MJ each having 2×2 pixels asillustrated in FIG. 9. Furthermore, it is assumed that the dividedpicture generating unit 11 divides each of the blocks M0, M1, MJ intosub-blocks B0, B1, B2, and B3 pixel by pixel. It is assumed that adivided picture P0 is obtained by collecting pixels of top-leftsub-blocks B0 from M0, M1, MJ divided in this manner, a divided pictureP1 is obtained by collecting pixels of top-right sub-blocks B1therefrom, a divided picture P2 is obtained by collecting pixels ofbottom-left sub-blocks B2 therefrom, and a divided picture P3 isobtained by collecting pixels of bottom-right sub-blocks B3 therefrom.

Here, an example in which a block of 2×2 pixels is divided intosub-blocks each having 1×1 pixel will be described. However, the sizesof a block and a sub-block are not limited to those of this example, anda similar implementation can be performed even when the presentinvention is applied to a larger size of the block or a larger size ofthe sub-block.

[Process of First Example]

In the first example described above, the following ultradivided-picture encoding and inter divided-picture encoding areperformed on a divided picture divided as in FIG. 9.

-   -   Process 1-1: Intra divided-picture encoding is performed on a        divided picture P0.    -   Process 1-2: A predicted picture for a divided picture P1 is        generated by applying an interpolation filter to a decoded        picture of the divided picture P0 using the decoded picture of        the divided picture P0 as a reference picture and inter        divided-picture encoding is performed. A decoded picture P1′ of        the encoded divided picture P1 is generated and stored. Its        manner is illustrated in FIG. 10A.    -   Process 1-3: A predicted picture for a divided picture P2 is        generated by applying an interpolation filter to the decoded        picture of the divided picture P0 using the decoded picture of        the divided picture P0 as a reference picture and inter        divided-picture encoding is performed. A decoded picture P2′ of        the encoded divided picture P2 is generated and stored. Its        manner is illustrated in FIG. 10B.    -   Process 1-4: For the divided pictures P1 and P2, sums S1 and S2        of prediction errors (e.g., a sum of absolute values or sum of        square errors of the prediction errors) between the predicted        pictures of the divided pictures P1 and P2 and the decoded        pictures P1′ and P2′ are calculated and compared with each        other.    -   Process 1-5: when S1≦S2.

As illustrated in FIG. 10C, the decoded picture of the divided pictureP2 is used as a reference picture in inter divided-picture encoding ofthe divided picture P3. That is, the inter divided-picture encoding ofthe divided picture P3 is performed by applying an interpolation filterto the decoded picture of the divided picture P2 to generate a predictedpicture of the divided picture P3 and encoding prediction errors betweenthe predicted picture and the divided picture P3. This is because acorrelation between pixels of the horizontal direction on the originalpicture is considered to be higher than a correlation between pixels ofthe vertical direction due to the fact that S1≦S2.

-   -   Process 1-6: when S1>S2.

As illustrated in FIG. 10D, the decoded picture of the divided pictureP1 is used as a reference picture in inter divided-picture encoding ofthe divided picture P3. That is, the inter divided-picture encoding ofthe divided picture P3 is performed by applying an interpolation filterto the decoded picture of the divided picture P1 to generate a predictedpicture of the divided picture P3 and encoding prediction errors betweenthe predicted picture and the divided picture P3. This is because acorrelation between pixels of the vertical direction on the originalpicture is considered to be higher than a correlation between pixels ofthe horizontal direction due to the fact that S1>S2.

[Process of Second Example]

In the second example described above, intra divided-picture encodingand inter divided-picture encoding are performed on the divided picturesdivided as in FIG. 9 as follows.

-   -   Process 2-1: Intra divided-picture encoding is performed on a        divided picture P0.    -   Process 2-2: A predicted picture for a divided picture P1 is        generated by applying an interpolation filter to a decoded        picture of the divided picture P0 using the decoded picture of        the divided picture P0 as a reference picture and inter        divided-picture encoding is performed. At this time, a sum S1 of        prediction errors is stored.    -   Process 2-3: A predicted picture for a divided picture P2 is        generated by applying an interpolation filter to the decoded        picture of the divided picture P0 using the decoded picture of        the divided picture P0 as a reference picture and inter        divided-picture encoding is performed. At this time, a sum S2 of        prediction errors is stored.    -   Process 2-4: The sums S1 and S2 of the prediction errors of the        divided pictures P1 and P2 are compared with each other.    -   Process 2-5: when S1≦S2.

A decoded picture of the divided picture P2 is used as a referencepicture in inter divided-picture encoding of the divided picture P3.That is, the inter divided-picture encoding of the divided picture P3 isperformed by applying an interpolation filter to the decoded picture ofthe divided picture P2 to generate a predicted picture of the dividedpicture P3 and encoding the prediction errors between the predictedpicture and the divided picture P3.

-   -   Process 2-6: when S1>S2.

A decoded picture of the divided picture P1 is used as a referencepicture in inter divided-picture encoding of the divided picture P3.That is, the inter divided-picture encoding of the divided picture P3 isperformed by applying an interpolation filter to the decoded picture ofthe divided picture P1 to generate a predicted picture of the dividedpicture P3 and encoding the prediction errors between the predictedpicture and the divided picture P3.

[Picture Decoding Apparatus]

FIG. 11 is a diagram illustrating an example of a configuration of thepicture decoding apparatus. The picture decoding apparatus 20 includesan information source decoding unit 21, an intra divided-picturedecoding processing unit 22, an inter divided-picture decodingprocessing unit 23, a decoded picture combining unit 24, a correlationdirection calculating unit 25, and a reference picture selecting unit26.

The picture decoding apparatus 20 inputs encoded data of a picturesubjected to compressive encoding by the picture encoding apparatus 10illustrated in FIG. 1. The information source decoding unit 21 performsentropy decoding on the input encoded data.

The intra divided-picture decoding processing unit 22 performs decodingon encoded data of at least predetermined one divided picture subjectedto intra divided-picture encoding in accordance with intradivided-picture prediction. The inter divided-picture decodingprocessing unit 23 decodes a decoding target divided picture inaccordance with inter divided-picture prediction using a decoded dividedpicture as a reference picture. The divided pictures decoded by theintra divided-picture decoding processing unit 22 and the interdivided-picture decoding processing unit 23 are input to the decodedpicture combining unit 24. The decoded picture combining unit 24generates a decoded picture by arranging each sub-block of the decodeddivided pictures at an original position on the original picture.

When the number of candidates for the reference picture is only one, forexample, when the number of decoded divided pictures to which a pixelnearest to a pixel position on the original picture of the decodingtarget divided picture belongs is one, the inter divided-picturedecoding processing unit 23 performs inter divided-picture decodingusing the decoded divided picture as a reference picture.

When there are a plurality of candidates for the reference picture, forexample, when there are a plurality of decoded divided pictures to whicha pixel nearest to the pixel position on the original picture of thedecoding target divided picture belongs, the correlation directioncalculating unit 25 obtains a divided picture in which a sum of absolutevalues or sum of squares of prediction errors is smallest among thedecoded divided pictures serving as the candidates for the referencepicture, determines, from the result, a direction of correspondingpixels of the decoded divided picture and its reference picture on theoriginal picture as a direction in which the correlation is high, andnotifies the reference picture selecting unit 26 of the correlationdirection. The process to be performed by the correlation directioncalculating unit 25 is exactly the same as that to be performed by thecorrelation direction calculating unit 15 in the picture encodingapparatus 10.

The reference picture selecting unit 26 selects a decoded dividedpicture in the correlation direction calculated by the correlationdirection calculating unit 25 as a reference picture for the decodingtarget divided picture and notifies the inter divided-picture decodingprocessing unit 23 of the reference picture.

[Flow of Picture Decoding Process]

FIG. 12 is a flowchart of the picture decoding process. The flow of thepicture decoding process will be described in accordance with FIG. 12.

First, the information source decoding unit 21 performs entropy decodingon input encoded data of a decoding target (step S401). Next, the intradivided-picture decoding processing unit 22 performs intradivided-picture decoding using a conventional intra-frame predictivedecoding method or the like such as that performed in H.264 for somedivided pictures P0 to PM (where M<N) among predetermined dividedpictures P0 to PN based on the input encoded data (step S402).

Subsequently, the inter divided-picture decoding processing unit 23performs inter divided-picture decoding on predetermined dividedpictures P(M+1) to PL using a decoded divided picture predetermined foreach divided picture as a reference picture (step S403).

Subsequently, the inter divided-picture decoding processing unit 23performs inter divided-picture decoding on divided pictures P(L+1) to PNthat are not yet decoded using a decoded divided picture in a directionhaving a high spatial correlation selected by the reference pictureselecting unit 26 based on the prediction errors of the decoded dividedpicture as a reference picture (step S404).

Finally, the decoded picture combining unit 24 combines pixels(sub-blocks) of divided pictures decoded by the intra divided-picturedecoding processing unit 22 and the inter divided-picture decodingprocessing unit 23 and outputs as a decoded picture (step S405).

[Example (Example 1) of Detailed Configuration of Picture DecodingApparatus]

FIG. 13 illustrates the first example of the detailed configuration ofthe picture decoding apparatus 20 illustrated in FIG. 11. Because theinformation source decoding unit 21, the decoded picture combining unit24, and the reference picture selecting unit 26 in the picture decodingapparatus 20 illustrated in FIG. 13 correspond to those of the samereference numerals illustrated in FIG. 11, a description thereof isomitted.

An intra divided-picture decoding unit 201 performs intradivided-picture decoding on divided pictures P0 to PM from decodinginformation of the divided pictures P0 to PM decoded by the informationsource decoding unit 21 and stores decoded pictures in a decoded picturememory 202. In the decoded picture memory 202, a decoded picture of adivided picture subjected to inter divided-picture decoding is alsostored later.

In order to perform inter divided-picture predictive decoding on adivided picture that is not yet decoded, a predicted picture generatingunit 203 generates a predicted picture for every divided picture block(hereinafter may be simply referred to as a divided picture) of thedivided picture using a decoded divided picture within the decodedpicture memory 202 as a reference picture. In generation of thepredicted picture, a predicted picture is generated by applying, to thereference picture, a predetermined filter determined by the relativeposition between corresponding pixels of the divided picture serving asa current decoding target and the reference picture on an originalpicture.

A prediction error decoding unit 204 decodes prediction errors of thedivided picture serving as a target of the inter divided-picturedecoding. In a picture decoding unit 205, an adder 206 adds thepredicted picture generated by the predicted picture generating unit 203to the prediction errors decoded by the prediction error decoding unit204 to generate a decoded picture. It is to be noted that, in thepicture decoding unit 205, a post-processing filter such as a deblockingfilter may be applied after the predicted picture is added to theprediction errors. This decoded picture is sent to the decoded picturecombining unit 24 and stored in the decoded picture memory 202.

A subtractor 207 subtracts each pixel value of the predicted picturegenerated by the predicted picture generating unit 203 from that of adivided picture block decoded by the picture decoding unit 205 andnotifies a prediction error calculating unit 208 of a subtractionresult. The prediction error calculating unit 208 calculates a sum ofprediction errors for every decoded divided picture serving as acandidate for the reference picture. A prediction error comparing unit209 obtains a divided picture in which the sum of the prediction errorscalculated by the prediction error calculating unit 208 is smallest,determines, from the result, a direction of corresponding pixels of thedecoded divided picture having the smallest sum of the prediction errorsand its reference picture on the original picture as a direction inwhich the correlation is high, and notifies the reference pictureselecting unit 26 of the correlation direction.

The reference picture selecting unit 26 selects a decoded dividedpicture in the correlation direction calculated by the prediction errorcomparing unit 209 from the decoded picture memory 202 as a referencepicture for the decoding target divided picture, and notifies thepredicted picture generating unit 203 of the reference picture.

[Example (Example 2) of Detailed Configuration of Picture DecodingApparatus]

FIG. 14 is the second example of the detailed configuration of thepicture decoding apparatus 20 illustrated in FIG. 11. Because thecomponents of the picture decoding apparatus 20 illustrated in FIG. 14having the same reference numerals as those of the first exampleillustrated in FIG. 13 described above have the same functions as thoseillustrated in FIG. 13, a detailed description thereof is omitted.

In the case of the second example, a prediction error calculating unit221 calculates a sum of prediction errors decoded by the predictionerror decoding unit 204 for every decoded divided picture (block)serving as a candidate for the reference picture. A prediction errorcomparing unit 222 obtains a divided picture in which the sum of theprediction errors calculated by the prediction error calculating unit221 is smallest, determines, from the result, a direction ofcorresponding pixels of the decoded divided picture having the smallestsum of the prediction errors and its reference picture on the originalpicture as a direction in which the correlation is high, and notifiesthe reference picture selecting unit 26 of the correlation direction.The reference picture selecting unit 26 selects a decoded dividedpicture in the correlation direction calculated by the prediction errorcomparing unit 222 from the decoded picture memory 202 as a referencepicture for the decoding target divided picture, and notifies thepredicted picture generating unit 203 of the reference picture.

[Specific Example of Decoding Process of First Example]

An example in which intra divided-picture decoding and interdivided-picture decoding are performed on four divided pictures P0 to P3obtained by rearranging pixels of blocks each having 2×2 pixels as inFIG. 9 described above will be described as the specific example of thedecoding process of the first example.

-   -   Process 1-1: Intra divided-picture decoding is performed on a        divided picture P0.    -   Process 1-2: A predicted picture for a divided picture P1 is        generated by applying an interpolation filter to a decoded        picture of the divided picture P0 using the decoded picture of        the divided picture P0 as a reference picture and inter        divided-picture decoding is performed. A decoded picture P1′ of        the decoded divided picture P1 is stored.    -   Process 1-3: A predicted picture for a divided picture P2 is        generated by applying an interpolation filter to the decoded        picture of the divided picture P0 using the decoded picture of        the divided picture P0 as a reference picture and inter        divided-picture decoding is performed. A decoded picture P2′ of        the decoded divided picture P2 is stored.    -   Process 1-4: For the divided pictures P1 and P2, sums S1 and S2        of prediction errors (e.g., a sum of absolute values or sum of        square errors of the prediction errors) between the predicted        pictures of the divided pictures P1 and P2 and the decoded        pictures P1′ and P2′ are calculated and compared with each        other.    -   Process 1-5: when S1≦S2.

The decoded picture of the divided picture P2 is used as a referencepicture in inter divided-picture decoding of the divided picture P3.That is, the inter divided-picture decoding of the divided picture P3 isperformed by applying an interpolation filter to the decoded picture ofthe divided picture P2 to generate a predicted picture of the dividedpicture P3 and adding the predicted picture to prediction errors of thedivided picture P3. This is because a correlation between pixels of thehorizontal direction on the original picture is considered to be higherthan a correlation between pixels of the vertical direction due to thefact that S1≦S2.

-   -   Process 1-6: when S1>S2.

The decoded picture of the divided picture P1 is used as a referencepicture in inter divided-picture decoding of the divided picture P3.That is, the inter divided-picture decoding of the divided picture P3 isperformed by applying an interpolation filter to the decoded picture ofthe divided picture P1 to generate a predicted picture of the dividedpicture P3 and adding the predicted picture to the prediction errors ofthe divided picture P3. This is because a correlation between pixels ofthe vertical direction on the original picture is considered to behigher than a correlation between pixels of the horizontal direction dueto the fact that S1>S2.

[Process of Second Example]

In the second example described above, intra divided-picture decodingand inter divided-picture decoding are performed on the divided picturesdivided as in FIG. 9 as follows.

-   -   Process 2-1: Intra divided-picture decoding is performed on a        divided picture P0.    -   Process 2-2: A predicted picture for a divided picture P1 is        generated by applying an interpolation filter to a decoded        picture of the divided picture P0 using the decoded picture of        the divided picture P0 as a reference picture and inter        divided-picture decoding is performed. At this time, a sum S1 of        prediction errors is stored.    -   Process 2-3: A predicted picture for a divided picture P2 is        generated by applying an interpolation filter to the decoded        picture of the divided picture P0 using the decoded picture of        the divided picture P0 as a reference picture and inter        divided-picture decoding is performed. At this time, a sum S2 of        prediction errors is stored.    -   Process 2-4: The sums S1 and S2 of the prediction errors of the        divided pictures P1 and P2 are compared with each other.    -   Process 2-5: when S1≦S2.

A decoded picture of the divided picture P2 is used as a referencepicture in inter divided-picture decoding of a divided picture P3. Thatis, the inter divided-picture decoding of the divided picture P3 isperformed by applying an interpolation filter to the decoded picture ofthe divided picture P2 to generate a predicted picture of the dividedpicture P3 and adding the predicted picture to prediction errors of thedivided picture P3.

-   -   Process 2-6: when S1>S2.

A decoded picture of the divided picture P1 is used as a referencepicture in inter divided-picture decoding of the divided picture P3.That is, the inter divided-picture decoding of the divided picture P3 isperformed by applying an interpolation filter to the decoded picture ofthe divided picture P1 to generate a predicted picture of the dividedpicture P3 and adding the predicted picture to prediction errors of thedivided picture P3.

It is to be noted that, in the above-described embodiments, one or moredivided pictures serving as a target of intra divided-picture encoding(the same is also applied to decoding) may be provided for one frame,and an encoding process in intra divided-picture encoding and interdivided-picture encoding may be performed in units of divided pictureblocks which are small areas obtained by dividing a divided picture.When a process of inter divided-picture encoding is performed in unitsof divided picture blocks, a reference picture may be switched based ona comparison of sums of prediction errors in units of divided pictureblocks.

[Example of Moving-Picture Encoding Apparatus to which Picture EncodingApparatus is Applied]

FIG. 15 illustrates an example of the moving-picture encoding apparatusto which the present invention is applicable. In the moving-pictureencoding apparatus 300, the present invention can be particularlyapplied to an encoding process associated with an intra frame predictingunit 301. The other components are similar to the configurations ofconventional general moving-picture encoding apparatuses used asencoders of H.264 or the like.

The moving-picture encoding apparatus 300 inputs an encoding targetvideo signal, divides a frame of the input video signal into blocks,performs encoding on every block, and outputs its bitstream as anencoded stream. For this encoding, a prediction residual signalgenerating unit 303 obtains a difference between the input video signaland a prediction signal which is an output of the intra frame predictingunit 301 or an inter frame predicting unit 302 and outputs it as aprediction residual signal. A transform processing unit 304 performs anorthogonal transform such as a discrete cosine transform (DCT) on theprediction residual signal and outputs transform coefficients. Aquantization processing unit 305 quantizes the transform coefficientsand outputs quantized transform coefficients. An information sourceencoding unit 311 performs entropy encoding on the quantized transformcoefficients and outputs an entropy encoding result as the encodedstream.

On the other hand, the quantized transform coefficients are also inputto an inverse quantization processing unit 306 in which inversequantization is performed. An inverse transform processing unit 307performs an inverse orthogonal transform on transform coefficients whichare an output of the inverse quantization processing unit 306, andoutputs a decoded prediction residual signal.

In a decoded signal generating unit 308, a decoded signal of an encodingtarget block which has been encoded is generated by adding the decodedprediction residual signal to the prediction signal, which is the outputof the intra frame predicting unit 301 or the inter frame predictingunit 302. In order that the decoded signal is used as a referencepicture in the intra frame predicting unit 301 or the inter framepredicting unit 302, it is stored in a frame memory 309. It is to benoted that when the reference picture is referred to in the inter framepredicting unit 302, an in-loop filter processing unit 310 inputs apicture stored in the frame memory 309, performs a filtering process ofreducing coding distortion, and a picture subjected to the filteringprocess is used as the reference picture.

In the ultra frame predicting unit 301, the encoding processes of theintra divided-picture encoding and the inter divided-picture encodingdescribed in the embodiments of the present invention are performed. Theinformation source encoding unit 311 performs entropy encoding oninformation about a prediction mode, a motion vector, or the like set inthe intra frame predicting unit 301 or the inter frame predicting unit302, and outputs as an encoded stream.

[Example of Moving-Picture Decoding Apparatus to which Picture DecodingApparatus is Applied]

FIG. 16 illustrates an example of the moving-picture decoding apparatusto which the present invention is applicable. In the moving-picturedecoding apparatus 400, the present invention is particularly applicableto a decoding process associated with an intra frame predicting unit402. The other components are similar to configurations of conventionalgeneral moving-picture decoding apparatuses used as decoders of H.264and the like.

The moving-picture decoding apparatus 400 inputs an encoded streamencoded by the moving-picture encoding apparatus 300 described withreference to FIG. 15, performs decoding, and outputs a video signal of adecoded picture. For this decoding, an information source decoding unit401 inputs the encoded stream, performs entropy decoding on quantizationtransform coefficients of a decoding target block, and decodesinformation about intra-frame prediction and information aboutinter-frame prediction. In the intra frame predicting unit 402, thedecoding processes of the intra divided-picture decoding and the interdivided-picture decoding described in the embodiments of the presentinvention are performed.

An inverse quantization processing unit 404 inputs the quantizationtransform coefficients, performs inverse quantization thereon, andoutputs decoded transform coefficients. An inverse transform processingunit 405 performs an inverse orthogonal transform on the decodedtransform coefficients and outputs a decoded prediction residual signal.A decoded signal generating unit 406 adds the decoded predictionresidual signal to a prediction signal which is an output of the intraframe predicting unit 402 or an inter frame predicting unit 403, andgenerates a decoded signal of the decoding target block. This decodedsignal is stored in a frame memory 407 in order to use the decodedsignal as a reference picture in the intra frame predicting unit 402 orthe inter frame predicting unit 403. It is to be noted that when thereference picture is referred to in the inter frame predicting unit 403,an in-loop filter processing unit 408 inputs a picture stored in theframe memory 407 and performs a filtering process of reducing codingdistortion, and a picture subjected to the filtering process is used asthe reference picture.

[Configuration Example by Computer]

FIG. 17 illustrates an example of a configuration of hardware when thepicture encoding apparatus 10 of FIG. 1 is configured by a computer anda software program. The present system has a configuration in which acentral processing unit (CPU) 50 which executes the program, a memory 51such as a random access memory (RAM) which stores the program and datato be accessed by the CPU 50, a picture signal input unit 52 (which maybe a storage unit which stores a picture signal by a disc apparatus orthe like) which inputs an encoding target picture signal from a cameraor the like, a program storage apparatus 53 which stores a pictureencoding program 54 which is a software program for causing the CPU 50to execute a process of encoding an input picture in accordance with thepresent technique, and an encoded data output unit 55 (which may be astorage unit which stores encoded data by a disc apparatus or the like)which outputs the encoded data generated by the CPU 50 executing thepicture encoding program 54 loaded to the memory 51, for example, via anetwork, are connected by a bus.

FIG. 18 illustrates an example of a configuration of hardware when thepicture decoding apparatus 20 of FIG. 11 is configured by a computer anda software program. The present system has a configuration in which aCPU 60 which executes the program, a memory 61 such as a RAM whichstores the program and data to be accessed by the CPU 60, an encodeddata storage unit 62 (which may be an input unit via a network or thelike) which inputs and stores encoded data encoded by the pictureencoding apparatus 10 of FIG. 1 in accordance with the presenttechnique, a program storage apparatus 63 which stores a picturedecoding program 64 which is a software program for causing the CPU 60to execute a process of decoding the encoded data in accordance with thepresent technique, and a decoded picture output unit 65 which outputs,to a reproduction apparatus or the like, a decoded picture which isobtained by the CPU 60 executing the picture decoding program 64 loadedto the memory 61 to perform decoding on the encoded data are connectedby a bus.

While embodiments of the present invention have been described abovewith reference to the drawings, it is apparent that the above-describedembodiments are examples of the present invention and the presentinvention is not limited by the above-described embodiments. Therefore,additions, omissions, substitutions, and other modifications ofstructural elements can be made without departing from the spirit orscope of the present invention.

INDUSTRIAL APPLICABILITY

The present invention is applicable to, for example, intra-framepredictive encoding and decoding. In accordance with the presentinvention, it is possible to improve the coding efficiency and reducethe computational complexity involved in a deblocking filter process.

DESCRIPTION OF REFERENCE SIGNS

-   10 Picture encoding apparatus-   11 Divided picture generating unit-   12 Intra divided-picture encoding processing unit-   13 Inter divided-picture encoding processing unit-   14 Information source encoding unit-   15, 25 Correlation direction calculating unit-   16, 26 Reference picture selecting unit-   20 Picture decoding apparatus-   21 Information source decoding unit-   22 Intra divided-picture decoding processing unit-   23 Inter divided-picture decoding processing unit-   24 Decoded picture combining unit

The invention claimed is:
 1. A picture encoding method for performingcompressive encoding on an input picture, the picture encoding methodcomprising: a divided picture generating step of, when the input pictureis divided into blocks each having n×m pixels and each divided block isdivided into sub-blocks each having n₁×m₁ pixels (where 1≦n₁<n and 1≦m₁<m), setting divided pictures of the same size including a set of pixelsof sub-blocks having the same relative position within the blocks; anintra divided-picture encoding step of performing intra divided-pictureencoding on at least one of the divided pictures; an interdivided-picture encoding step of selecting, among encoded dividedpictures, an encoded divided picture having the shortest distance on anoriginal picture with respect to pixels at the same position in anencoding target divided picture and the encoded divided pictures as areference picture, generating a predicted picture for the encodingtarget divided picture using the selected reference picture, andperforming inter divided-picture encoding; a correlation directioncalculating step of calculating a direction in which a correlation witha pixel on the original picture is highest with respect to pixels at thesame position in each encoded divided picture serving as a candidate forthe reference picture and a reference picture of each encoded dividedpicture serving as the candidate for the reference picture based on aprediction error in inter divided-picture prediction of each encodeddivided picture serving as the candidate for the reference picture whena plurality of candidates for the reference picture are present; areference picture selecting step of selecting an encoded divided picturein the direction in which the correlation is high for the encodingtarget divided picture as the reference picture when the plurality ofcandidates for the reference picture are present; and an informationsource encoding step of performing information source encoding onencoding results in the intra divided-picture encoding step and theinter divided-picture encoding step.
 2. The picture encoding methodaccording to claim 1, wherein, in the correlation direction calculatingstep, a predicted picture is generated from a decoded picture of eachencoded divided picture serving as a candidate for the reference pictureand a reference picture of each encoded divided picture, a sum ofprediction errors of the predicted picture for the decoded picture ofeach encoded divided picture is calculated, and a direction connectingpixels on the original picture corresponding to pixels at the sameposition in a combination of an encoded divided picture in which the sumof the prediction errors is smallest and a reference picture isdetermined as the direction in which the correlation is high.
 3. Thepicture encoding method according to claim 1, wherein, in thecorrelation direction calculating step, a sum of prediction errorscalculated in the inter divided-picture encoding is calculated for eachencoded divided picture serving as a candidate for the referencepicture, and a direction connecting pixels on the original picturecorresponding to pixels at the same position in a combination of anencoded divided picture in which the sum of the prediction errors issmallest and a reference picture is determined as the direction in whichthe correlation is high.
 4. The picture encoding method according toclaim 1, 2, or 3, wherein, in at least the inter divided-pictureencoding step, the correlation direction calculating step, and thereference picture selecting step, a process of encoding the dividedpictures, a process of calculating a correlation direction, and aprocess of selecting the reference picture are performed for each ofpicture blocks into which each divided picture is divided, and thereference picture in the inter divided-picture encoding is switched forevery picture block.
 5. A picture decoding method for performingdecoding on encoded data of a picture subjected to compressive encodingin which, when an input picture is divided into blocks each having nxmpixels and each divided block is divided into sub-blocks each havingn₁×m₁ pixels (where 1≦n₁ <n and 1≦m₁<m), divided pictures of the samesize including a set of pixels of sub-blocks having the same relativeposition within the blocks are set and encoding for every dividedpicture is performed, the picture decoding method comprising: aninformation source decoding step of inputting the encoded data of thepicture subjected to the compressive encoding and performing informationsource decoding; an intra divided-picture decoding step of performingintra divided-picture decoding on at least one of the divided picturesfrom data decoded in the information source decoding step; an interdivided-picture decoding step of selecting, among decoded dividedpictures, a decoded divided picture having the shortest distance on anoriginal picture with respect to pixels at the same position in adecoding target divided picture and the decoded divided pictures as areference picture, generating a predicted picture for the decodingtarget divided picture using the selected reference picture, andperforming inter divided-picture decoding; a correlation directioncalculating step of calculating a direction in which a correlation witha pixel on the original picture is highest with respect to pixels at thesame position in each decoded divided picture serving as a candidate forthe reference picture and a reference picture of each decoded dividedpicture serving as the candidate for the reference picture based on aprediction error in inter divided-picture prediction of each decodeddivided picture serving as the candidate for the reference picture whena plurality of candidates for the reference picture are present; areference picture selecting step of selecting a decoded divided picturein the direction in which the correlation is high for the decodingtarget divided picture as the reference picture when the plurality ofcandidates for the reference picture are present; and a decoded picturecombining step of combining a decoded picture from a divided picturedecoded in the intra divided-picture decoding step and a divided picturedecoded in the inter divided-picture decoding step.
 6. The picturedecoding method according to claim 5, wherein, in the correlationdirection calculating step, a predicted picture is generated from eachdecoded divided picture serving as a candidate for the reference pictureand a reference picture of each decoded divided picture, a sum ofprediction errors of the predicted picture for each decoded dividedpicture is calculated, and a direction connecting pixels on the originalpicture corresponding to pixels at the same position in a combination ofa decoded divided picture in which the sum of the prediction errors issmallest and a reference picture is determined as the direction in whichthe correlation is high.
 7. The picture decoding method according toclaim 5, wherein, in the correlation direction calculating step, a sumof prediction errors calculated in the inter divided-picture decoding iscalculated for each decoded divided picture serving as a candidate forthe reference picture, and a direction connecting pixels on the originalpicture corresponding to pixels at the same position in a combination ofa decoded divided picture in which the sum of the prediction errors issmallest and a reference picture is determined as the direction in whichthe correlation is high.
 8. The picture decoding method according toclaim 5, 6 or 7, wherein, in at least the inter divided-picture decodingstep, the correlation direction calculating step, and the referencepicture selecting step, a process of decoding the divided picture, aprocess of calculating a correlation direction, and a process ofselecting the reference picture are performed for each of picture blocksinto which each divided picture is divided, and the reference picture inthe inter divided-picture decoding is switched for every picture block.9. A picture encoding apparatus which performs compressive encoding onan input picture, the picture encoding apparatus comprising: a dividedpicture generating unit which, when the input picture is divided intoblocks each having nxm pixels and each divided block is divided intosub-blocks each having n₁×m₁ pixels (where 1≦n₁<n and 1≦m₁ <m), setsdivided pictures of the same size including a set of pixels ofsub-blocks having the same relative position within the blocks; an intradivided-picture encoding unit which performs intra divided-pictureencoding on at least one of the divided pictures; an interdivided-picture encoding unit which selects, among encoded dividedpictures, an encoded divided picture having the shortest distance on anoriginal picture with respect to pixels at the same position in anencoding target divided picture and the encoded divided pictures as areference picture, generates a predicted picture for the encoding targetdivided picture using the selected reference picture, and performs interdivided-picture encoding; a correlation direction calculating unit whichcalculates a direction in which a correlation with a pixel on theoriginal picture is highest with respect to pixels at the same positionin each encoded divided picture serving as a candidate for the referencepicture and a reference picture of each encoded divided picture servingas the candidate for the reference picture based on a prediction errorin inter divided-picture prediction of each encoded divided pictureserving as the candidate for the reference picture when a plurality ofcandidates for the reference picture are present; a reference pictureselecting unit which selects an encoded divided picture in the directionin which the correlation is high for the encoding target divided pictureas the reference picture when the plurality of candidates for thereference picture are present; and an information source encoding unitwhich performs information source encoding on encoding results by theintra divided-picture encoding unit and the inter divided-pictureencoding unit.
 10. The picture encoding apparatus according to claim 9,wherein the correlation direction calculating unit generates a predictedpicture from a decoded picture of each encoded divided picture servingas a candidate for the reference picture and a reference picture of eachencoded divided picture, calculates a sum of prediction errors of thepredicted picture for the decoded picture of each encoded dividedpicture, and determines a direction connecting pixels on the originalpicture corresponding to pixels at the same position in a combination ofan encoded divided picture in which the sum of the prediction errors issmallest and a reference picture as the direction in which thecorrelation is high.
 11. The picture encoding apparatus according toclaim 9, wherein the correlation direction calculating unit calculates asum of prediction errors calculated in the inter divided-pictureencoding for each encoded divided picture serving as a candidate for thereference picture, and determines a direction connecting pixels on theoriginal picture corresponding to pixels at the same position in acombination of an encoded divided picture in which the sum of theprediction errors is smallest and a reference picture as the directionin which the correlation is high.
 12. The picture encoding apparatusaccording to claim 9, 10, or 11, wherein at least the interdivided-picture encoding unit, the correlation direction calculatingunit, and the reference picture selecting unit perform a process ofencoding the divided pictures, a process of calculating a correlationdirection, and a process of selecting the reference picture for each ofpicture blocks into which each divided picture is divided, and switchthe reference picture in the inter divided-picture encoding for everypicture block.
 13. A picture decoding apparatus for performing decodingon encoded data of a picture subjected to compressive encoding in which,when an input picture is divided into blocks each having n×m pixels andeach divided block is divided into sub-blocks each having n₁×m₁ pixels(where 1≦n₁ <n and 1≦m₁<m), divided pictures of the same size includinga set of pixels of sub-blocks having the same relative position withinthe blocks are set and encoding for every divided picture is performed,the picture decoding apparatus comprising: an information sourcedecoding unit which inputs the encoded data of the picture subjected tothe compressive encoding and performs information source decoding; anintra divided-picture decoding unit which performs intra divided-picturedecoding on at least one of the divided pictures from data decoded bythe information source decoding unit; an inter divided-picture decodingunit which selects, among decoded divided pictures, a decoded dividedpicture having the shortest distance on an original picture with respectto pixels at the same position in a decoding target divided picture andthe decoded divided pictures as a reference picture, generates apredicted picture for the decoding target divided picture using theselected reference picture, and performs inter divided-picture decoding;a correlation direction calculating unit which calculates a direction inwhich a correlation with a pixel on the original picture is highest withrespect to pixels at the same position in each decoded divided pictureserving as a candidate for the reference picture and a reference pictureof each decoded divided picture serving as the candidate for thereference picture based on a prediction error in inter divided-pictureprediction of each decoded divided picture serving as the candidate forthe reference picture when a plurality of candidates for the referencepicture are present; a reference picture selecting unit which selects adecoded divided picture in the direction in which the correlation ishigh for the decoding target divided picture as the reference picturewhen the plurality of candidates for the reference picture are present;and a decoded picture combining unit which combines a decoded picturefrom a divided picture decoded by the intra divided-picture decodingunit and a divided picture decoded in the inter divided-picture decodingunit.
 14. The picture decoding apparatus according to claim 13, whereinthe correlation direction calculating unit generates a predicted picturefrom each decoded divided picture serving as a candidate for thereference picture and a reference picture of each decoded dividedpicture, calculates a sum of prediction errors of the predicted picturefor each decoded divided picture, and determines a direction connectingpixels on the original picture corresponding to pixels at the sameposition in a combination of a decoded divided picture in which the sumof the prediction errors is smallest and a reference picture as thedirection in which the correlation is high.
 15. The picture decodingapparatus according to claim 13, wherein the correlation directioncalculating unit calculates a sum of prediction errors calculated in theinter divided-picture decoding for each decoded divided picture servingas a candidate for the reference picture, and determines a directionconnecting pixels on the original picture corresponding to pixels at thesame position in a combination of a decoded divided picture in which thesum of the prediction errors is smallest and a reference picture as thedirection in which the correlation is high.
 16. The picture decodingapparatus according to claim 13, 14 or 15, wherein at least the interdivided-picture decoding unit, the correlation direction calculatingunit, and the reference picture selecting unit perform a process ofdecoding the divided picture, a process of calculating a correlationdirection, and a process of selecting the reference picture for each ofpicture blocks into which each divided picture is divided, and switchthe reference picture in the inter divided-picture decoding for everypicture block.
 17. A non-transitory computer-readable medium for storinga picture encoding program for causing a computer to execute the pictureencoding method according to claim 1, 2, or
 3. 18. A non-transitorycomputer-readable medium for storing a picture decoding program forcausing a computer to execute the picture decoding method according toclaim 5, 6, or 7.